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Xanadu Light, March 94. 


© 1994 Theodor Holm Nelson, Xanadu™ On-Line Publishing, 
3020 Bridgeway #295, Sausalito CA 94965. Tel. 415/ 331-4422, fax 
415/ 332-0136. 


"Xanadu," "Xanadu Light," "HyPerformance" and the Eternal 
Flaming X symbol are trademarks of Project Xanadu. 


This is an unpolished and preliminary introduction to Xanadu Light, 
especially its table and data structures. I assume that the reader 
knows in general what this is about. 


OVERVIEW 


Xanadu has long been a software design, an ideal, and an action 
plan for a worldwide linked electronic publishing system. 

(Note that the system was also designed from the beginning for 
version and document management on a personal and corporate 
scale, but version management has been dropped from the current 
system. We expect to put in a form of version management at a 
later time.) 


Openness and Freedom 
We believe the Xanadu system offers unique solutions to 
issues of royalty, rights and re-use. 
Our electronic publishing model has been unique: a scheme 
for open, integrated universal electronic publishing with 
freedom for anyone to publish. Anyone may publish any 
form of data, and anyone may publish links to already- 
published material. 


Mix-and-Match Boilerplate 
The system is defined so that all parts of a published 
document may be purchased a la carte in pieces. Anything 
already published within the system may also be re-used and 
re-published with ease within the system. Thus text, 
graphics, audio, animations, simulations may be recombined 
on a mix-and-match basis. This is true for links as well. 


Re-Use 
All parties agree by contract that materials may be re-used 
freely, but only by virtual inclusion in new Xanadu 
documents. Thus everyone is free to use previous materials 
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in new ways, but royalty and credit automatically go to the 
rightsholder. 


Clean Data 
Links and markers are local applicative substructures, and 
not embedded in any way. This better facilitates re-use of 
the material. 


Automatic Royalty 
Each time a customer buys a fragment of any data-- text, 
graphics, audio, 3D models, whatever-- a proportional 
royalty payment automatically goes from the customer to the 
publisher. Anything may be bought in small portions. 


Licensed Service Providers 
The plan has also been to license, or franchise, service 
providers, who will set up computers and data storage using 
our software, and make the service available to customers 
locally and internationally. Customers may modem in to 
their local service providers. The service providers, in turn, 
will be able to send for materials they do not have locally. 


Really a Business Model 
Essentially this is a business model, and the type of 
implementation is secondary. 


Xanadu Light 
XANADU LIGHT is the new implementation now being 
planned for implementation with conventional database on 
the Internet. 


What follows is a design specification for that Xanadu Light. 


00. THE BUSINESS MODEL 


The service provider is not responsible for the contents. 
The publisher is responsible for contents, and pays for storage. 
The reader/user pays for delivery, including both 
a service charge to the service provider, and 
a royalty to the publisher. This royalty is exactly 
proportional to the number of bytes, at the royalty price set 
by the publisher on that document, or set by the publisher on 
a particular segment of that document. 
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THE CONTRACTS 
Contracts bind the system. 
Publisher -- 
is responsible for the contents and relinquishes context 
control of the materials in the Xanadu network in 
return for royalty and credit. Publisher retains all 
other rights. 


Reader -- 
agrees to terms of purchase, and agrees not to 
redistribute the materials except through transclusion 
into documents published within the Xanadu network. 


Service provider -- 
agrees to maintain integrity of the document, use the 


software provided, and remand royalties and other 
payments as called for. 


0. THE MODEL OF INTERACTION 


The user perceives a point-and-click universe. In actuality, each 
click makes another small purchase of content bytes or links from 
the network. 
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bought and assembled by 
front-end screen machine C may be kept if user wants Publisher 
THE RECEIPT 


A receipt will be sent with each portion. This will assist the 
user, or the user's system, 1n filing the portion, if the user 
chooses to keep it. The receipt will also authenticate, by 
means of a hash code, the user's legitimate purchase of this 
portion. 


CLIENT SOFTWARE 
Basic purchase will be simple, allowing a minimal interface 
for purchasing portions. A higher-performance interface 
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will be point-and-click with material appearing in separate 
Mac-style windows. 


RECOMMENDED LAYOUT 
There have been various complex arrangements (especially 
SGML) for the framing and presentation of text, especially 
local assignment of layouts to paragraph types. We will 
have such an arrangement, but within the framework of local 
applicative substructures. 


THE HIGH-PERFORMANCE CLIENT 
A very high-performance interface will also be specified at a 
later time. 
For a higher level of performance, with panning, scrolling 
and zooms, as well as other effects, we will be specifying at 
a later time a high-performance client language called 
HyperFormance™, This will be closely related to multilayer 
compositing methods in video, and extend the video EDL 
(Edit Decision List). 


1. GENERAL MODEL OF A DOCUMENT. 
A document consists of contents bytes (text, graphics, etc., 
ORIGINATING IN THIS DOCUMENT), and connections both in 
and out of the document. 
The structure document connections is APPLICATIVE, meaning 
that no codes are embedded. Structure is managed within the 
database system. 


CONTENT BYTES J 
Connections 
OUTBOUND 


Connections 
INTERNAL 


In relation to other documents, these connections create a 
symmetrical structure of connections: 
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CONTENT BYTES 
Connections Connections 
INBOUND OUTBOUND 















cinieitan | > 
cinieitan | > 






All connections are owned. One document's inbound connection is 
another document's outbound connection. 


ee BYTES 
Connections 
OUTBOUND 
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cine, | > 










ANOTHER DOCUMENT 


Connections 
INBOUND 









Content 
Bytes 


Connections into one document are owned by other documents 
where they originate. Taken all together, these connected 
documents make up a docuverse. 


i — 





OUT 


hg 


2. TYPES OF CONNECTIONS 
There are two types of connections: links and transclusions. 
A link is an arbitrary connection of any type. 
A transclusion means a new instance: “bring in this material here," 
even though place of origin is different. 
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TYPES OF CONNECTION 
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A link is a connector, of arbitrary extensible type, between 
part of a document and another part of a document. The link 
may be within or between documents. 


TYPES OF LINKS 
There can be many types of link, in an ever-extensible list. 
A short beginning list: 
Subject (what commentator thinks this is about) 
Comment 
Disagreement 
Endorsement 
Evidence 
Example 
Endorsement 
Graphical illustration 


LINK ENDSETS 
The endset is the material a link is attached to at either end 
(eg text, graphic, “hot button”). 
In Xanadu Light, there will be many types of endsets. These 
types will simply tell the system where the link is considered 
to be attached. 


ENDSET COUPLINGS 
The endset coupling specifies how it is attached: to a point, 
to a span of bytes, an area of picture, etc. Various different 
coordinate-spaces may be defined for this coupling. 
Couplings can also be Deep or Surface. Deep couplings 
reach to data structure, whereas surface couplings reach the 
result of projecting the data structure. Example: a deep 
coupling to a PostScript document would couple to text 
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bytes inside the original PostScript code, a surface coupling 
would couple merely to an area of the output surface 
generated by the PostScript program. 


LINK DIRECTIONALITY, VISIBILITY, FOLLOWABILITY 
All Xanadu links can be seen from either end and followed 
from either end. Some people call this "bidirectional." This 
is incorrect. All our links are Divisible (can be seen from 
both ends) and bifollowable (can be followed from both 
ends). 

Links are usually directional, that is, asymmetrical in 
meaning. Very few link types are symmetrical ("is related 
to" would be one possible example). 


TRANSCLUSIONS 
A transclusion means "bring this material in here." The 
formal definition: virtual inclusion across a document 
boundary. 
The material is not copied into the new document, but 
always newly-purchased from the original, or a functioning 
instance of the original. 
Content bytes, such as text and graphics, may be 
transcluded. 

transclusion of content bytes 


MS GYyy- 


Links themselves may be transcluded. In practice this means 
that a connector of a certain type, and either its from -set or 
its to-set is transcluded. 


transclusion of link WW 


DIRECTIONS OF A TRANSCLUSION 


Transclusions, like links, are bivisible and bifollowable. 
Transclusions are directional. To simplify things we say that 
a transclusion that brings in material is an outbound 
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connection. But there are of course two directions: the 
direction in which the material has been chosen, or the 
transclusion has been created (the direction of creation and 
choice), and the direction in which the material flows to be 
included (the direction of inclusion). 


DIRECTIONS OF A TRANSCLUSION 


direction 
of creation 


and choice SS. 
SW 


= 
inclusion 





3. STORAGE IN XANADU LIGHT 
There will be two types of storage in Xanadu Light: content files, 
which may be anything that can be stored in a computer file, and 
database tables, which are the means by which the system will 
keep track of document structures. 
Before considering the distinctions between content files and 
database tables, however, we will consider the logic of storage and 
payment. 


4. EXTENDED STORAGE and the PAYMENT 


MODEL 


THE PUBLISHER'S COMMITMENT 


At the time of publication, the publisher pays for published 
materials to be stored on at least three nodes for at least three 
years. 

This applies to contents bytes and required tables, including 
name tables, piece tables, outgoing connection tables, 
harpoon tables, and other tables that the evolving system 
may eventually require. 

(These charges are not expected to be great.) 


WHAT'S STORED TOGETHER LOCALLY 


While the Xanadu docuverse is a logical unity, in fact it will 
be dispersed, and so it must be designed to perform as if it 
were all in one place, at least insofar as possible. 
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For this reason, various copies of materials will be stored 
together as needed. 

Payment for these specific storages is part of the system 
design. 


HARPOON TABLES for INBOUND CONNECTIONS 
Because inbound pointers come from many other 
documents, copies of them need to be collected with the 
document they point to. This collection of inbound 
connections is the “harpoon table." 


= CONTENT BYTES C je 
as aga onnections 
| vent —— OUTBOUND 
ee at 
i Connections 
Inbound 
Connections 
INTERNAL 


Payment for the storage of the harpoons is by the respective 
publishers of those connections. 

This is not optional. All published outgoing connections 
require the storage of an inbound harpoon. 


WHAT'S STORED TOGETHER 












Consolidated 
Harpoon Tables 










Copies of all published 
inbound connections 







BARNACLE STORAGE OF CONNECTED MATERIAL 
A publisher may choose to store transcluded material with 
the document which requires them. This transcluded 
material is a local copy of parts of the other document, and 
remains the intellectual property of the first publisher. 
The new publisher pays for barnacle storage, but all royalties 
for the transclusion are passed on to the original publisher. 
The new publisher gains the advantage of being able to use 
the other material freely, though without direct royalty 
benefit. 
Barnacle storage of a subdocument is logically just like any 
other document: it has its own piece table and connection 
tables. 
Barnacle storage will be an option both for inbound 
connected material and outbound connected material. 
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CONTENT 
BYTES 


Content bytes Connections Inbound et aS 
= _- OUTBOUND 
Connec- Connections Inbound 
Conneéstions 
INTERNAL 


OUTBOUND BARNACLES 


INBOUND BARNACLES 
















barnacle: Consolidated 







parts of Harpoon Tables 
other Copies of all published 
document inbound connections 










Consolidated barnacle: 
Harpoon Tables parts of 
Copies of all published other 

inbound connections document 
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SLOSH 
"Slosh"refers to the service provider making copies of 
documents, or parts of documents, for the convenience of 
providing survice. It is anticipated that such copies will be 
made and destroyed freely, hence they will "slosh" 
throughout the system. 
No royalty is paid for slosh copies, which are strictly for 
improved service. Royalties will be paid in the usual 
manner when materials are sent from slosh copies to end 
users. 





OTHER SPECIAL CHARGES 
SITE LICENSING 

A number of sites, such as corporations and libraries, 
wish to make a fixed payment for a document, rather 
than maintaing per-use charges. For such site 
licensing, an actuarial method will be used to estimate 
the probahle revenues from a document at that site. 
Note that publishers are not required to consent to site 
licensing; site licensing is offered as an additional 
rider on the publishing contract which a publisher may 
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decline. This is because the per-use payment model is 
the most fundamental aspect of Xanadu publishing, 
and we do not wish to confuse it with other 
requirements. 


FIRST-PUB PREMIUM 
A portion of the service charge, including some 
minimum, will be remanded to the node of first 
publication whenever deliveries are made from a 
document. This provides additional incentive to 
service providers to acquire published material. 


5. REPRESENTING THIS MODEL IN DATABASE 
TABLES 


Note that this is an approximate listing of requirements. Avoiding 
redundancy and reducing it to sensible sets of contents are both 
critical. 


THE DOCUMENT TABLES: OVERVIEW 


Recall the simple model of the document: contents, internal 
links and markers, external links and markers. 








DOCUMENT 


CONTENTS 
HARPOON TABLE: 


Incoming connections Outgoing connections 


More fully, this consists of various specific types of 
connections in addition to the contents bytes. 
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DOCUMENT 










HARPOON TABLE: OUTGOING 







Incoming connections CONNECTIONS 
from all over 
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Internal Connections 


Transclusions 


Links 
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other documents) 


Transclusions é 
Links 
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this document) 
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Links 








In order explicitly to represent this as a system of database 
tables, a variety of tables are needed. 

The link and transclusion tables are meant to be handled by 
identical mechanism. The other tables are separate. 


DOCUMENT ITSELF 


Storage paid by publisher of this document. 







HARPOON 

TABLES 
Storage paid by 
various publishers 








OUTBOUND TABLE 


LINKS 
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LINKS AND 


INBOUND 






PIECE 


LINKS 





p 
rd 
OF CON- | 7 
, 
, 





















é 
: ’ 
: 
é 
, | TABLE 
MARKERS ‘ f All TENTS 
é 
' : contents 
y : (User- p 
j in compre- , CON- 
/ virtual hensible) : TENE 
, 
INBOUND INTERNAL | ‘ |ouTBoUND |, | 24¢ress Author, ‘FILES 
TRANS- TRANS- , | TRANS- fa tadetaks freebies , 
CLUSIONS CLUSIONS | * | CLUSIONS | 4 | (Repetitions (“dust j 
(external + &externals jacket"), / 
uses of this é = trans- section é 
material) / — clusions) titles, F 
4 indexes ...) 4 


(NOT SHOWN: Internet title tables for anonymous FTP, to 
be paid for by publishers.) 


THE DOCUMENT TABLES: FURTHER 


DETAIL 
THE TABLE OF CONTENTS 
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This shows the sections of a document that are of 
significance to the author and user, as distinct from 
the details of their origin, location and cost. 


THE PIECE TABLE 


The piece table is the internal technical structure 
which lists the location and ownership of the pieces of 
the document. They are listed according to the 
address sequence of a document; this address 
sequence may not refer to sequence of delivery or 
viewing, especially in a hypertext or hypermedia 
document. 
The pieces may be stored in separate files in a single 
directory on the home node, if they are new material; 
they may be arbitrary portions of files on the home 
node; or they may be files or arbitrary portions of files 
elsewhere. 
A piece has at least the following information: 
Piece or Segment 

Ordinal position 

Location (THIS NODE OR 

ELSEWHERE) 

Copyright owner 

Publisher on Xanadu 

(If not on Xanadu, where) 

(If not on Xanadu, terms of availability) 

Royalty Rate 


TRANSCLUSION TABLES 


Note that the outbound transclusion table is derived 
from the piece table, which lists pieces regardless of 
where they are stored. If they are not stored in the 
logical space of this document, they are transclusions. 
Transclusion tables should all be in the same format: 


Piece 
Data type 
Length 
Copyright owner 
Publisher 
Royalty rate per byte 
LINK TABLES 


Link tables should all be in the same format: 
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Endset 
Left endset type 
Right endset type 
Left endset coupling 
(point, span, coordinate space, 
surface vs. deep, etc.) 
Right endset coupling 
Where 
Where left endset applies: DocID, 
Address 
(implies location and length) 
Where right endset applies: DocID, 
Address 
(implies location and length) 
What 
Data type 
Length 
Copyright owner 
Publisher 
Royalty rate per byte 


THE ACCOUNTING TABLES 


CUSTOMER HISTORY TABLE 
Charges for private use 


Payment for document deliveries 

Storage of customer's private files 

Table space for private database-managed 
documents 

Service charges for private database-managed 
documents 

ROYALTY PAYMENTS OUT 

Modem connect time 


Charges for Xanadu publishing 


Publication registry fees 

Content file storage 

Service charges for database management 
(local) 

Service charges for database management 
(remote) 

Table space for customer's Xanadu-published 
database-managed documents (local and 
remote) 
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ROYALTY PAYMENTS IN (on customer's 
Xanadu-published documents) 


CUSTOMER HOLDINGS TABLE 
This table is stored at the customer's home node. It 
keeps track of the customer's privately stored 
materials and published materials. 


Customer's publications 
Customer's Xanadu-published filenames 
Customer's Xanadu-published filespace 
Table space of published documents 
Document-management tables for published 
documents 


Customer's private files and documents 
Private filespace 
Private filenames 
Document-management tables for private 
documents 
Table space of unpublished documents 


DOCUMENT BUSINESS TABLE 
(Accessible to document owner only.) 
Note that any records kept of who has bought material 
will only be for preliminary debugging purposes. It is 
mandatory that in the long run such records NOT be 
kept.) 
Usage history tables are maintained with barnacle 
copies, and are forwarded to the publisher's home 
node from time to time. 


This contains: 
Date of publication 


Storage charages for harpoon-table entries 
Storage charges for barnacle files 

Storage charges and storage payments, by 
segment 


Publishing registration costs 
Storage, monthly 
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Down payment for three years 


Bytes sold and royalty receipts, by segment 
(A "segment" is a section of native data 
of a particular type, or having a specific 
royalty setting) 

Transaction: amount and time 
Transaction: amount and time 
Transaction: amount and time 


6. GENERAL DIALOG OF INTERACTION 


The user will send for contents bytes or a set of links. 

The user will pay for that delivery (including royalty on the 
contents bytes). 

The system will deliver the content bytes or links. 

The system will deliver a receipt, which serves as a re-entry ticket. 


7. SESSION THREAD (advanced system) 


In the advanced protocol, the user will present identification and 
get a session thread ticket. The ticket permits re-entry to the 
session without re-identification or re-establishment of credit. 
After the first interaction, each receipt serves as the next ticket. 


8. THE RECEIPT (advanced system) 


Whenever a portion of material is sent out to a customer, a receipt 
is sent along with the portion to identify that portion. The user's 
system may store the receipt with the portion in order to keep 
acquired materials manageable. 

The receipt is hashed against the piece as a form of authentication. 
The purchaser may present this receipt as proof of the purchase if 
there are disputes on billing. 

Receipts are demonstrably consecutive. 

The receipt also provides authentication of the data, through hash 
codes. However, the hash is also against the purchase information, 
so that the material can only be authenticated along with who 
purchased it and when. 








